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Abstract. In the case of graph partitioning, the emergence of localized eigenvectors 
can cause the standard spectral method to fail. To overcome this problem, the 
spectral method using a non-backtracking matrix was proposed. Based on numerical 
experiments on several examples of real networks, it is clear that the non-backtracking 
matrix does not exhibit localization of eigenvectors. However, we show that localized 
eigenvectors of the non-backtracking matrix can exist outside the spectral band, which 
may lead to deterioration in the performance of graph partitioning. 


1. Introduction 

Graph partitioning or community detection is a popular topic in the held of complex 
networks and computer science; the spectral method is a standard heuristic for it. 
Because the graph partitioning problems are usually formulated as discrete optimization 
problems, which are NP problems, we require some heuristics. In the spectral method, 
the discreteness constraint is relaxed, and the problem is formulated as a continuous 
optimization problem. The simplihcation then reduces to an eigenvalue/eigenvector 
problem of a matrix (see [1] for detail). 

The commonly used matrices in the literature are the unnormalized Laplacian 
L = D — A, normalized Laplacian C = [1], and modularity matrix 

M = {A- kk^/K) /K [2l[3], where A is the adjacency matrix; each element represents 
a node pair and Aij = 1 if the node pair i and j are connected by a link, otherwise 
Aij = 0. We only consider undirected graphs and thus A is symmetric. The matrix D 
is a diagonal matrix whose element is equal to the node’s degree ki = Aij. In the 
modularity matrix M, fc is a column vector of degrees, k = {ki, ..., k]\f)'^ (T represents 
the transpose), and K = the total degree. The spectral method utilizes the 

eigenvectors of large or small eigenvalues; for example, in the case of bisection using L 
and £, the signs of eigenvector elements of the second smallest eigenvalue determines 
the module assignment of nodes. That is, the nodes with the same sign belong to 
the same module. On the other hand, for bisection using M, the eigenvector of the 
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largest eigenvalue is referred. Though the result of the spectral method obtained by the 
abovementioned procedure often leads to a good approximation of the true optimum, 
there is no theoretical guarantee that this is always true. 

In addition to the matrices specihed earlier, recently, much attention has been paid 
to the spectral method using the non-backtracking matrix B [1] , which is defined by the 
following 2N x 2N matrix: 


B 


0 D-l\ 

-I A J ’ 


( 1 ) 


where / is an identity matrix. The matrix B is asymmetric and its spectrum is complex. 
For graph partitioning, however, one usually focuses on large real eigenvalues. The left 
eigenvector of ([I]) has the eigenvector of the form (-u"'", where fi is an eigenvalue, 

and we refer to the sign of v of the second-largest eigenvalue for module assignment in 
the bisection problem. 

The emergence of localized eigenvectors causes the spectral method to fail. A 
localized eigenvector is one whose weight is concentrated around a few components while 
the remaining components have zero weights or weights close to zero. Furthermore, the 
localized eigenvector has no information of global community structure. For example, 
if a localized eigenvector is the leading eigenvector of the modularity matrix M, the 
standard bisection method provides a solution that is uncorrelated to the true optimum. 
Indeed, for sparse stochastic block models, localized eigenvectors emerge frequently 
around the detectability threshold when we use the adjacency matrix A [1] and the 
Laplacians, L and C [5] ; this has a considerable effect on the consistency with the planted 
partition. The details of the stochastic block model are discussed below. The spectral 
partitioning method using the non-backtracking matrix B was originally introduced to 
avoid localization in such a situation [1]. 

In this paper, first, we briefly review previous works on the localized eigenvector 
(Sec. [2]). We then conduct a comparative analysis of the degree of localization for 
the abovementioned matrices in several real networks and show that non-backtracking 
matrix does not exhibit high inverse participation ratios (IPRs) (Sec. [3]). Based on this 
result, it may be assumed that the eigenvectors of the non-backtracking matrix never 
exhibit localization. However, in Sec. lU we show that localized eigenvectors with real 
eigenvalues do exist; furthermore, one of them may have the second-largest eigenvalue 
(Sec. [5]). Finally, some discussion on the localized eigenvectors is given in Sec. [71 


2. Degree of localization 


The degree of localization can be measured by the IPR. For an eigenvector v, the IPR 
is defined as follows. 


IPR = 


EX 


(Exr 

The value of IPR is close to 1 if t) is localized and is of 0{N~^) for a graph with 
size if -y is an extended vector. Although localized eigenvectors have been studied 
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for decades, there is still much scope for a more thorough theoretical understanding 
[6l[7llHl[9l[5l[ini[IIl[l2]. The localized eigenvectors of adjacency matrix A in complex 
networks that exist owing to the hub structure have been frequently discussed in the 
literature [HI [m [n]; in addition, it is known that the localized eigenvectors of the 
modularity matrix M also exists because of the hub structure [lO]. However, it is 
noteworthy that this is not always the case, depending on the matrix one considers |15j . 
Furthermore, there are many researches on the localized eigenvectors in real networks as 
well [m CHI [IT]. To the best of our knowledge, the comparative analysis of localization 
for commonly used matrices has not been performed thoroughly. 

3. Degree of localization in real networks 

We investigate the performance of the spectral method of matrices used for graph 
partitioning in real networks. We used the datasets present in [THl CH]. The networks 
we consider may not have clear module structures, and for simplicity, we ignore the 
direction in case of directed networks; further, we deleted self loops if they exist, and 
converted multiple links to a single link. 

In Fig. d we plotted the largest IPR of the 6 smallest or largest eigenvectors of 
each matrix. We plotted the small eigenvalues for the unnormalized and normalized 
Laplacians, L and £, and the large (real) eigenvalues for the modularity and non¬ 
backtracking matrices, M and B. Note that the non-backtracking matrix may have less 
than 6 real eigenvalues. In many networks, the IPRs of unnormalized and normalized 
Laplacian, L and C, can be very large, while those of the modularity matrix M tend 
to be very small. In particular, the non-backtracking matrix B never exhibits a high 
IPR. It is interesting to note that the modularity matrix M is robust to localization 
as the non-backtracking matrix B very often. Concerning the degree of localization, in 
practice, it can be inferred that localization can rarely be problematic when M is used. 

4. Localized eigenvectors of the non-backtracking matrix 

We have observed that the eigenvectors of non-backtracking matrix are quite robust 
against localization. It might be expected that the non-backtracking matrix exhibits 
high IPRs when many short loops exist. However, it is hard to obtain high IPRs by 
random generation of triangles (see Appendix). A question that needs to be answered 
is whether it can be proven that the non-backtracking matrix never has a localized 
eigenvector. Although it may be extremely rare in real networks, localized eigenvectors 
do exist in the non-backtracking matrix. 

It is known that, for the random-walk Laplacian, I — A, localized eigenvectors 
can be constructed owing to the existence of a pair of symmetric subgraphs. The 
construction of a graph with such a local symmetry is called motif doubling [31]. The 
procedure of motif doubling is as follows. Let H be a set of nodes of a subgraph (or 
a motif) in a graph GiV^E), where V and E denote the sets of nodes and links of 
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Figure 1. (Color online) The values of largest IPR among 6 eigenvectors with the 
largest or smallest eigenvalues in 11 real networks. First 3 networks, adjnoun [3], 
Political books [2], Political blogs [20] are the datasets distributed by Mark Newman 
[18] . Other networks, ca-Astro [21] . Phca-HepTh |21| . cit-HepPh |221 123]. wiki-Vote 
(SUES), p2p-Gnutella04 [HIES], oregonl-010331 email-Enron |28lE9], and loc- 
brightkite [30] are of SNAP [19]; the datasets are labeled as listed in m- 


the entire graph, respectively. We denote the neighbors of O by dQ. We assnme that 
O is only a small part of the entire graph, i.e., |r2| <C \V\. We bnild a copy of the 
induced snbgraph of O, whose node set is denoted as O. For each node ia C there 
exists a corresponding node i^ G hi. We then connect this copied snbgraph to the nodes 
in dQ in the same manner as the nodes in O do. That is, if ia is connected to some 
nodes in dfl, ia is also connected to those particular nodes (see Fig. [2] for example). 
In [3l], it is shown that the entire graph that is generated by the motif doubling has a 
right eigenvector of the random-walk Laplacian whose component is concentrated within 
hi U hi. For its explicit form, see EH or refer to our derivation for the non-backtracking 
matrix below. Although [3T| considers the procedure of doubling as explained above, 
hereafter, we regard that this locally symmetric subgraph naturally exists in the graph 
that we consider. 

We can apply the motif doubling to obtain the eigenvectors of the non-backtracking 
matrix B. Its eigenvalue equation is Av = p~^{D — I)v, where p is the eigenvalue. 

We restrict ourselves to the case where the induced subgraph of hi is a regular graph 
and every node in hi is connected to the nodes in l/\(hlUhl) by a single link (see Fig. [2]), 
because it makes the eigenvalue equation signihcantly simple. We hrst consider the 
eigenvalue equation for hlUF'(hl, 9hl), the induced subgraph of hi, and the links between hi 
and dkl. Denoting its corresponding eigenvector as we have the eigenvalue equation 


as 



(3) 


Note that, although the nodes in ([3]) are of hi, the degree of each node is not c — 1, but 
c, because ih(hl, dkl) is included. Using we can construct a localized eigenvector v 
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Figure 2. 


(Color online) An example of motif doubling with a clique of 3 nodes. 


of the entire graph. Its components are proportional to 


Vi = 


.,n 


for i = ia E n, 
for i = ia E fl, 
otherwise. 


(4) 


It can be easily conhrmed that this is indeed an eigenvector of the entire graph. Though 
the nodes in V\dQ trivially satisfy the eigenvalue equation with p, for a node g E dQ, 
using (jH), we have 


i€V 




gi 


^/i + 


——"j ^ Agi ^ AijVj + 



c — 1 
h 


= 0 . 




Therefore, the vector of (jl]) satishes the eigenvalue equation of the entire graph, yielding 


IPR 


1 


( 6 ) 


In the following sections, we demonstrate the emergence of the localized eigenvector 
(jH) using synthetic graphs and show that its eigenvalue can indeed be the second-largest 
one. 


5. A pair of cliques attached to a stochastic block model 

As a hrst example, let us consider a clique of n nodes as and a sparse stochastic block 
model as the base graph l/\(r2 Uf2). The neighboring nodes of the clique dQ are chosen 
randomly. The stochastic block model is constructed as follows. We denote the total 
number of nodes N and each node has its planted module assignment r, i.e., for the two 
planted module case, r = 1,2. Every node pair is connected at random on the basis of 
their module assignments. Here, we consider a fundamental stochastic block model of 
two modules with an assortative structure; a node pair is connected with probability pin 
if they belong to the same module, and with probability Pout (Pout < Pin) if they belong 
to different modules. That is, the nodes in the same module are more densely connected 
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Figure 3. (Color online) (a) Fraction of correctly classified nodes and (b) the IPR as 
a function of the strength of block structure Cjn — Cout for the sparse stochastic block 
models with a pair of cliques. As the stochastic block model, we set N = 1000 for two 
equal size modules and the average degree c = 3; the detectability threshold of the non¬ 
backtracking matrix is at Cjn — Cout = 2-\/3 ~ 3.46. In both plots, circles represents the 
result of the standard stochastic block model, while the crosses, triangles, diamonds, 
and squares represent the cases where the clique pair with n = 4, 5, 6, 7 are attached, 
respectively. The plots show the average over 100 samples with error bars. In (b), 
the solid lines indicate the values of IPRs equal to 1/12, 1/14, and 1/A^. The inset 
of (a) represents the eigenvector elements of the second-largest eigenvalues of a single 
realization. For this particular case, we set N = 200 and Cin — Cout = 4, n = 6. The 
first n nodes indicate the randomly chosen nodes in and the last 2n nodes indicate 
the nodes in 17 and 17. 


than the node in other modules. In order to impose the constraint that the graph is 
sparse, we set bothpin and Pout to be of 0(l/iV), i.e., the average degree remains constant 
as we increase the total number of nodes N. Following the notation in the literature, let 
us introduce variables Cin = Pin-^ = 0(1) and Cout = PoutN = 0(1). Because each node 
pair is connected independently randomly, the graph has a Poisson degree distribution; 
further, because the graph is sparse, it is locally tree-like. As the block structure gets 
weaker, i.e., Cin — Cout is smaller, the result of the spectral method loses the correlation 
with the planted solution. The value at which they become completely uncorrelated is 
called the detectability threshold [32l |33l HJ |5]; the region above this threshold is the 
detectable phase, whereas region below it is the undetectable phase. 

We now solve for ([3]). In the present case, the degree of each node is n. If we 
consider the factor /i-|- (n — l)//i as another constant A, it just represents an eigenvalue 
equation for the adjacency matrix of the induced subgraph of f/. The eigenvalues of a 
clique are —1 and n — 1. Because we are interested in an eigenvector that is out of the 
spectral band, we solve for the solution with a real eigenvalue. For A = n — 1, we have 

g? — [n — l)/i -b n — 1 = 0, (7) 

V(^- l)(^-5)) • (8) 

Therefore, the eigenvalue can be real for n > 5. For the solution with A = —1, we 
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Figure 4. (Color online) (a) Fraction of correctly classified nodes, (b) IPR, and 
(c) second-largest eigenvalue as a function of the degree c in for various values of 
cq. As the stochastic block model, we set N = 10000 for two equal size modules and 
Cin ~ Cout = 2co — 1, i.e., the base graph has a very clear block structure. We set 
|f2| = 50, so that the value of the IPR for the eigenvector of ([4]) is 0.01, which is 
indicated as a solid line in (b). The plots show the average over 20 samples with error 
bars. In (c), solid line indicates the eigenvalue /r in dH]). 


always have a complex eigenvalue for /i. Note that, a real eigenvalue does not always 
indicate that it is out of the spectral band. Moreover, it may not be the second-largest 
eigenvalue. As we demonstrate below, however, it can actually be the eigenvector with 
the second-largest eigenvalue. 

For the stochastic block model, it is known that the spectral method of the non¬ 
backtracking matrix B is free of localization [1]. On the other hand, when the clique 
pair is attached as described in motif doubling, as shown in Fig. |3](a), the eigenvector 
of the second-largest eigenvalue can be uncorrelated to the planted partition even 
in the region where the block structure is fairly strong. According to (IH]), we have 
/i ~ 3.618 for n = 6 and it is out of the edge of the spectral band \/h ~ 1.732 of the 
standard stochastic block model. Figure [3](b) indicates that such spectral partitions 
exhibit high IPRs, which coincide with {2n)~^ for n > 5. Therefore, we conclude that 
the eigenvector we considered in the previous section can indeed be the one with the 
second-largest eigenvalue. Although the eigenvector of n = 5 is also uncorrelated, it is 
extended. Interestingly, Fig. [3](b) shows that the eigenvectors with high IPRs gradually 
appear when n = 4 as the block structure gets weaker. Accordingly, the decrease of 
correlation with the planted partition hastens. However, we note that this eigenvector 
has nonzero weights for the components corresponding to the nodes outside the cliques 
and is different from the eigenvector in (l4j) . 

6. A pair of regular graphs attached to a regular graph 

To show that setting a clique for H is not essential, we demonstrate another example. 
We consider a random (c — l)-regular graph as the induced subgraph of H and a random 
Co-regular graph with block structure as the base graph I/\(H U H). As mentioned in 
Sec. m each node in H is connected to a node in the base graph by a single link; i.e.. 
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the degree of a node in is c. The eigenvalue problem of the present case contains 
the previous example as a special case. The leading eigenvalue for is c — 1 and the 
eigenvalue jx of the non-backtracking matrix is 



Figure 0] shows the results of numerical experiments and the estimates for the 


localized eigenvector of (01). They show that, when c > 6 and c > cq +1 are satisfied, we 
can obtain the localized eigenvector of (j3]) as the one with the second-smallest eigenvalue, 
which is analogous to the previous example. 

7. Discussion 

For commonly used matrices for graph partitioning and the non-backtracking matrix, 
we conducted comparative analyses of the eigenvector localization for several examples 
of real networks. The non-backtracking matrix did not show high values of IPR in any 
of these analyses. We also observed that the modularity matrix M is as robust against 
localization as the non-backtracking matrix B in many cases. Although the localized 
eigenvector of the non-backtracking matrix seems rare in practice, we constructed an 
example that exhibits localization. 

The eigenvector of (0]) is particularly interesting because it is strongly localized; the 
eigenvector elements are exactly zero outside the pair of motifs due to the symmetry. 
In general, the elements of a localized eigenvector far from the center of localization do 
not need to be exactly zero. As we observed in Sec. |5l the eigenvector of (0j) is not the 
only one that exhibits a high IPR. However, the forms of eigenvectors of other types 
may depend on global quantities, e.g., size of the entire graph. Thus, it is more difficult 
to evaluate their IPRs. Moreover, having a high IPR is not sufficient to conclude that 
the eigenvector is uninformative with respect to global structure. 

Even when we have the localized eigenvector of (0j), it may be possible to interpret 
that the spectral method correctly identihed the symmetric pair of subgraphs as 
a characteristic structure. However, this eigenvector obtained through the spectral 
method is not an indicator that we expect to obtain in graph partitioning; the 
eigenvector of (0]) does not convey any information about the module assignment of 
the nodes outside of hi U dkl. Therefore, considering that, the information regarding 
global structure is lost. 

The existence of localized eigenvectors out of the spectral band implies that the 
standard procedure that utilizes q leading eigenvectors for q modules may fail in practical 
situations. Note, however, that it does not necessarily mean that the detection is 
impossible; by searching more eigenvectors, one might find informative eigenvectors. 
To the best of our knowledge, the condition of the possible case is an open question for 
any matrix. 
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Appendix A. Degree of localization in the stochastic block model with 
triangles 

The dynamic interpretation of the non-backtracking matrix B is that, when we regard 
B as a. transition matrix, it prevents a walker from traveling back on the link it has just 
traveled. On the other hand, localization can be understood as the phenomenon that 
the walker visit a few characteristic nodes very frequently in the case of the adjacency 
matrix. Hence, it makes sense that the non-backtracking matrix avoids localization in 
sparse random graphs, because short loops are rare. This motivated us to analyze how 
it performs for the graphs with many short loops. Because there are many short loops 
in real networks even though the graph is globally sparse, this setting is also interesting 
for other matrices. 

We investigate whether localized eigenvectors emerges in random graphs with many 
triangles. Our goal is to verify if a larger number of short loops tend to destroy or 
enhance the localized eigenvector. To observe this tendency, let us consider perturbing 
a sparse random graph so that the number of triangles is increased. As the base random 
graph, we again use a sparse stochastic block model with two planted modules (see 
the description in the main text for the details of the stochastic block model). The 
simplest algorithm to increase the number of triangles in a graph is to add some links. 
However, such algorithm changes the degree distribution simultaneously and makes it 
difficult for us to study the effect solely by short loops. For the same reason, we also 
wish to avoid increasing the degree correlation as much as we can. To this end, we 
employ the algorithm proposed in [Mj. In the algorithm of [M], links are randomly 
rewired so that the number of triangles increases, while keeping the degree sequence of 
the original graph. Although there are many centralities that measure the degree of 
clustering, because we just need to know the degree of perturbation, we simply count 
the number of triangles increased, i.e., Ar compared with the base graph and normalize 
it by N. Although the algorithm allows the graph to be disconnected, we keep the graph 
connected. 

The results for the benchmark graphs are shown in Fig. lAli We plotted the values 
of the IPR of eigenvectors of the unnormalized Laplacian L, the normalized Laplacian 
£, the modularity matrix M, and the non-backtracking matrix B, respectively. For the 
Laplacians, L and £, short loops considerably enhance the localization, although for 
the unnormalized Laplacian L, the IPR is slightly suppressed as Ar increases in the 
region where the localized eigenvectors already exist before rewiring. As we observed in 
the analysis of real networks, the modularity matrix M is signihcantly robust against 
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localization. Although localized eigenvectors may emerge in a larger graph with more 
triangles, in the present case, the IPR is considerably small in all regions, and therefore 
we conclude that the localization problem is expected to be rare in the modularity 
matrix. Despite many short loops, the non-backtracking matrix B does not exhibit high 
IPRs. However, we may have larger values of IPRs in a graph with an even stronger 
clustering structure. 
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Figure Al. (Color online) The density plot of the IPR in stochastic block models 
with controlled number of triangles. The horizontal axis represents the strength of 
block structure c-m — Cout; the larger is this value, the stronger is the block structure. 
The vertical axis represents the number of triangles increased, i.e.. At by the rewiring 
algorithm, divided by the total number of nodes N. For the unnormalized and 
normalized Laplacians, L and C, the IPRs of eigenvectors with the second-smallest 
eigenvalues are plotted, whereas the IPRs of the eigenvectors with the largest and 
second-largest eigenvalues are plotted for the modularity matrix M and the non¬ 
backtracking matrix R, respectively. The plots show the average taken over 20 samples. 
As the base stochastic block model, we set two equal size modules, where the sum of 
their nodes is TV = 1000, and the average degree c = 5. 
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